View-Action Representation Learning for Active First-Person Vision

In visual navigation, a moving agent equipped with camera is traditionally controlled by an input action and the estimation of features from sensory state (i.e. view) treated as pre-processing step to perform high-level vision tasks. this paper, we present representation learning approach that, instead, considers both inputs. We condition encoded feature transition network on that changes view camera, thus describing scene more effectively. Specifically, introduce module generates decoded higher dimensional representations increase representational power. then fuse output intermediate response predicts future state. To enhance discrimination capability among predictions different actions, further triplet ranking loss $N$ -tuplet functions, which in turn can be integrated regression loss. demonstrate proposed reinforcement imitation learning-based mapless navigation tasks, where learns navigate only through performed action, without external information.